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Abstract 

A framework  for  planning  algorithms  that  is  both  optimal  and  complete  will  be  presented. 
The  algorithm  allows  for  the  planning  of  optimal  paths  through  a multi-level  hierarchy  of 
annotated  graph  spaces.  A rich  world  model  that  contains  multi-resolution  attributes, 
costs,  and  predicates  controls  the  integrated  incremental  construction  and  evaluation  of 
the  planning  graph  in  dynamic  environments.  The  properties  of  this  framework  are 
proven  and  a number  of  example  problem  domains  are  presented  that  show  how  this 
framework  can  incorporate  both  hard  and  soft  constraints  during  both  graph  construction 
and  arc  evaluation. 
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Introduction 

Many  of  today’s  challenging  planning  problems  may  be  solved  through  the  use  of 
discrete  representations.  These  include  such  diverse  problems  as  robotic  vehicle  planning, 
parts  assembly  planning,  and  domain  independent  planning  systems.  These  problems  may 
also  include  dynamic  environments  in  which  real  time  replanning  is  required  and  in 
which  novel  planning  and  multi-resolution  data  requirements  exist.  The  problem  space 
for  these  tasks  may  be  flat  (single  level)  or  may  involve  multilevel  hierarchies. 

Once  the  problem  has  been  mapped  to  a discrete  form,  a planning  graph  may  be  created 
and  evaluated  in  order  to  plan  sequences  of  actions  that  allow  the  system  to  meet  specific 
goals.  In  traditional  graph  planning  techniques,  this  planning  graph  is  fully  constructed 
beforehand  and  stored  for  use  during  the  planning  cycle.  In  this  paper,  a new  framework 
is  presented  that  incorporates  and  extends  existing  graph  planning  and  search  techniques 
and  that  allows  for  the  incremental  construction  and  evaluation  of  the  planning  graph 
during  the  execution  of  the  planning  cycle.  A rich  world  model  that  may  contain  data  at 
multiple  resolutions  and  in  multiple  formats  controls  this  incremental  construction  and 
evaluation. 

It  will  be  shown  that  this  framework  allows  for  significant  savings  in  both  time  and 
memory  requirement  for  the  planning  system.  In  addition,  the  plans  created  will  be  able 
to  take  into  account  both  constraints  on  the  possible  state  space  and  the  cost/benefit  of 
state  transitions.  Constraints  on  the  planning  space  strictly  prohibit  certain  state 
transitions,  while  cost/benefit  analysis  allows  the  system  to  prefer  certain  classes  of  state 
transitions  to  others. 


1 NIST,  stepheirgnist.gov 

2 Work  performed  while  guest  researcher  at  NIST  from  Center  for  Computing  Technologies.  University  of 
Bremen.  Germany,  herzog@;tzi.de 
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In  addition  to  finding  a set  of  state  transitions  that  allow  the  system  to  transition  from  the 
current  state  to  the  goal  state,  later  sections  will  show  that  this  framework  finds  plans  that 
are  the  optimal  set  of  transitions  (optimal  path)  necessary  to  traverse  from  the  system’s 
current  state  to  the  goal  state  with  respect  to  a dynamic  cost  function  and  system 
constraints.  In  this  case,  optimality  refers  to  the  property  that  given  a set  of  discrete  states 
and  possible  state  transitions,  no  lower  cost  walk  of  state  transitions  exists  than  the 
solution  provided  by  this  system  for  a given  cost  function.  The  properties  of  this 
framework  are  proven  and  a number  of  example  problem  domains  are  presented  that 
show  how  this  framework  can  incorporate  hard  and  soft  constraints  during  both  graph 
construction  and  arc  evaluation. 

Framework  Definitions 

It  is  desired  to  define  a planning  space  S that  will  allow  for  a cost  optimal  sequence  of 
transitions  to  be  enumerated  for  the  task  of  transitioning  from  one  element  of  S to 
another.  In  order  to  sequence  the  members  of  S,  an  ordered  sequence  T of  points  (ti, 

...,tm)  in  time  and  the  node  space  ND  will  now  be  defined.  The  space  S over  Nn,  is  a 
basic  structure  such  that  s^  e S where  k = (kp...,kn),  ki,...,kn  e N. 


Definition  1:  The  node  space  ND  is  defined  by  (SxT)  and  nd  e ND  = (s^tj)  , ti  e T. 


Figure  1:  Depiction  of  spaces  and  sets  from  problem  formulation. 


The  space  S for  n = 2 is  depicted  as  a plane  in  Figure  1 . The  space  ND  is  depicted  as  the 
partially  filled  spheres  that  lie  in  several  time  indexed  planes. 

Before  one  can  discuss  the  optimality  of  a path,  there  must  exist  attributes  with  which  to 
compute  the  cost/benefit  of  the  path.  Under  the  current  construction  of  the  node  space, 
the  only  cost/benefit  measure  that  can  exist  (and  hence  the  only  measure  with  which  to 
compute  optimality)  is  the  number  of  node  transitions  that  make  up  a plan.  An  additional 
objective  of  the  planning  system  is  to  provide  the  infrastructure  necessary  to  develop  a 
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more  complex  notion  of  cost.  In  order  to  accomplish  this,  the  notion  of  attributes  will  be 
introduced.  These  attributes  capture  all  of  the  static  and  time  varying  features  that  can  be 
associated  with  sr  at  time  t. 

k 


Definition  2:  Define  the  set  of  possible  node  general  attributes  NGATT  = {ngatt  | ngatt 
is  a possible  feature  of  nd^  t g ND}. 


It  is  possible  that  a specific  node  ndj-  ( will  not  contain  all  of  the  possible  general 

attributes.  Therefore  the  set  of  actual  node  attributes  NATT  c;  NGATT  will  be  defined. 
ATT  is  defined  as  a set  of  sets,  where  each  member  attr  is  the  set  NATT  of  all  features 

k,t  

that  can  possibly  be  associated  with  s^  at  time  t. 


Definition  3:  Define  the  set  of  actual  node  attributes  NATT  = {natt  | natt  is  a feature  of 
nd- 1 g ND}  and  the  set  of  attribute  sets  as  ATT  = { NATT^  1 1 k = (kx,...,kn),  ki,...,kn  g 

N.feT}. 

The  set  of  attributes  ATT  is  represented  in  Figure  1 as  a collocated  plane  (the  shaded 
regions)  where  each  shaded  rectangle  represents  a set  of  attributes  NATT.  To  simplify 
future  notation,  an  annotated  node  space  AND  may  be  defined  that  combines  the  node 
space  ND  and  the  set  of  attribute  sets  ATT. 


Definition  4:  An  annotated  node  space  AND  is  defined  as  AND  = (NDxATT)  where 
andr  g AND. 

k,t 


If  the  planning  space  is  feature  rich,  NATT  may  be  quite  large.  Computing  the 
cost/benefit  ratio  by  utilizing  all  of  the  features  in  this  potentially  large  set  will  needlessly 
increase  the  computational  burden.  Instead  of  utilizing  NATT,  a new  set  of  node  relevant 
features  NRATT  will  be  defined  that  only  includes  the  features  that  are  relevant  for  the 
current  instantiation  of  the  cost/benefit  evaluation  at  a given  time,  and  is  a subset  of 
NATT.  RATT  is  then  defined  as  the  set  of  sets  consisting  of  the  feature  sets  NRATT  and 
is  a subset  of  ATT. 


Definition  5:  Define  the  set  NRATT  = { nrattr  I nratt,-  <z  nattr  } to  be  the  set  of  all 
domain  dependent  node  relevant  attributes  for  node  nd^  t and  the  set  of  relevant 
attribute  sets  as  RATT  = {NRATTtt  | k = (kj,...,kn),  ki,...,kn  g N,  f g T }. 
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Figure  2 depicts  the  attribute  set  relationships  NRATT  <z  NATT  c:  NGATT  and  the  node 
relationships  RAND  c:  AND  c:  S along  with  the  functional  relationships  between 
attributes  and  nodes. 


Figure  2:  Concentric  areas  showing  the  set  relationships  between  the  attribute  sets  and 
nodes,  with  arcs  showing  the  relationships  between  them. 


It  is  possible  that  a given  s-  at  time  t will  not  posses  any  relevant  features  and 
nrattk  t will  be  empty.  The  nodes  corresponding  to  members  of  RATT  that  contain  non- 
empty NRATT  sets  are  depicted  as  the  white  rectangles  in  Figure  1,  i.e.  RATT  is  used  as 
a constraint  on  the  node  space  AND  in  order  to  create  a set  of  relevant  annotated  nodes. 


Definition  6:  The  set  of  relevant  annotated  nodes  RAND  cz  AND  is  defined  by  RAND  = 
{randk  t e RAND,  randkt  = (and£  t ,nratt£  t ) | ||nratt£  t ||  > 0,  k = (k19. . . ,kn) , ki,...,kn 

G N,  I = (l1,...,ln),  li,...,ln  G N,  t,  G T }. 


The  following  definitions  on  graphs  are  based  on  definitions  from  Aho  and  Ullman  [1]. 
In  order  to  form  a graph,  a set  of  connections  between  nodes  must  be  established.  This 
connection  is  from  a start  node  to  an  end  node,  and  may  be  viewed  as  a binary  relation 
between  the  two  nodes.  These  binary  relations,  or  arcs  are  depicted  as  black  arrows  in 
Figure  3. 


Figure  3:  A path  through  a planning  graph  and  the  spanning  set  for  each  node  in  the  path. 
The  path  origin  is  the  black  circle,  the  goal  is  the  white  circle,  and  the  intermediate  nodes 
are  the  gray  circles. 
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Definition  7:  An  arc  e ARC  is  defined  as  a binary  relation  r on  RAND.  A directed  graph 
consists  of  a set  of  nodes  RAND  and  a set  of  arcs  ARC. 


Given  a set  of  attributed  nodes,  a domain  dependent  spanning  function  may  be  defined 
that  specifies  which  nodes  are  reachable  from  a given  node  in  a single  transition.  These 
nodes  are  called  the  node’s  successors  and  are  connected  with  an  arc. 


Definition  8:  Define  the  spanning  set  SP  of  a node  rand-  as  the  set  of  all  successor 

k.tj 

nodes  SP(rand£  )={sp£  |||nratt£  ||>0}. 

The  spanning  set  of  each  black  node  at  time  t is  shown  for  the  next  consecutive  time  slice 
in  Figure  3.  The  series  of  black  nodes  and  the  arcs  that  connect  them  form  a path  through 
the  graph.  The  path  in  Figure  3 is  of  length  3.  It  should  be  noted  that  the  spanning  set  is 
time  dependent,  and  therefore  the  graph  topology  may  change  on  a planning  cycle  to 
planning  cycle  basis.  In  addition  the  spanning  set  for  node  nd-  at  time  t may  not  be  equal 

to  the  spanning  set  for  node  nd^  at  time  t+1. 


Definition  9:  Given  a directed  graph  GR  = (RAND,  ARC),  a path  prand(  rand-  , 

rand,  ),  k = (k1,...,kn),  ki,...,kn  e N,  I = (lls...,ln),  li ln  e N,  t,  e T,  is  defined  as 

a list  of  nodes  (v1>t  ,v2>t  ,...,vm  ) such  that  = k , vm  = 1 and  there  is  an  arc  from  each 

node  to  the  next.  The  length  m of  the  path  is  defined  as  the  number  of  arcs  along  the  path. 

Definition  10:  A graph  is  defined  as  an  acyclic  graph  if  no  path  begins  and  ends  at  the 
same  node. 

Lemma  1:  The  graph  GR  is  an  acyclic  graph. 

Proof:  Assume  that  the  path  prand(  rand,  , randT  ),  k = 1 exists.  Then  by  Definition  9 

Mo  1 -»t  o 

there  must  exist  an  arc  r(  rand,  , randT  ) such  that  d > 0 and  randT  is  a successor  of 

v J.t^d  ’ Mo'  Mo 

rand,  . However,  by  Definition  8,  rand,  g SP( rand,  ),  d > 0,  and  t + d > t. 

J>*0+d  Mo  J»*o+d 

Therefore  r(  rand,  , rand.  ) does  not  exist  and  prand(  rand-  , rand,  ),  k = 1 , 

J’^0+d  Mo  Mo 

cannot  exist  which  contradicts  the  basic  assumption. 


As  previously  stated,  one  of  the  objectives  of  this  system  is  not  to  find  simply  any  path 
from  a given  start  node  to  a given  goal  node,  but  to  find  a cost  optimal  path.  Since  a path 
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is  a series  of  arcs,  a function  for  computing  the  cost  of  an  arc  must  first  be  defined.  As 
stated  by  Fernandez  and  Gonzalez  [6],  this  function  is  known  as  a cost  function  and  may 
be  defined  as: 


Definition  11:  The  mapping  c:ARC— is  called  the  cost  function  of  an  arc. 


The  set  of  costs  C must  be  at  least  a partially  ordered  set,  but  may  also  be  a totally 
ordered  set  and  must  have  the  “plus”  (+)  operator  defined.  One  typical  specification  for 

the  set  (C  is  C = 91+. 

Once  the  cost  of  individual  arcs  is  known,  it  is  a simple  matter  to  compute  the  cost 
associated  with  an  entire  path. 


Definition  12:  The  cost  associated  to  a path  prand  is  defined  as  pc  = ^c(r(vi,vi+1)) 

i=l 

The  system  objective  of  finding  the  cost  optimal  path  between  two  nodes  may  now  be 
achieved  by  finding  the  minimum  cost  path. 

Definition  13:  Define  the  minimum  cost  path  prandmm  between  the  nodes  rand^  t and 
randf  t+m where  m^Oand  k =(k!,...,kn),  ki,...,kn  e N,  I = (l15...,ln) , li,...,ln  e N such 
that  pc(prandmm)  < pc(prand),  prand  e {q  e PRAND  | q = prand(rand].  t ,randf  )}. 


Integrated  Incremental  Graph  Generation  and  Planning 
Algorithm 

In  a traditional  graph  planning  approach,  a graph  that  represents  the  entire  discrete 
problem  space  is  constructed  prior  to  a search  [2,3,1 1,13,15],  The  problem  constraints  are 
then  encoded  as  weights  on  the  individual  arcs,  and  a graph  search  is  performed  to 
determine  the  minimum  cost  path  through  the  planning  space.  As  will  be  shown,  this  has 
potentially  unrealizable  memory  requirements  to  store  the  graph  structure  and 
unrealizable  computational  time  requirements  to  determine  graph  node  connections  and 
arc  costs.  In  addition,  as  stated  by  Russell  ([14],  p.  92),  the  only  place  where  knowledge 
can  be  applied  once  the  graph  has  been  constructed  is  in  the  choice  of  which  node  to 
expand  next. 

An  algorithm  will  now  be  presented  that  incrementally  builds  and  evaluates  a planning 
graph  in  order  to  find  the  minimum  cost  path  prandmm  between  the  nodes  rand — and 

start, t0 
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rand—  ( m . Unlike  traditional  graph  search  techniques  such  as  breadth  first,  depth  first, 

and  A*  [12,14],  this  technique  provides  for  knowledge  to  be  applied  in  both  the  graph 
construction  and  planning  step.  Only  as  much  of  the  graph  as  is  necessary  to  perform  the 
task  at  hand  will  be  constructed,  and  this  construction  will  be  interleaved  with  the  arc  cost 
evaluation  and  graph  search.  Through  the  use  of  this  technique,  both  hard  and  soft 
constraints  on  the  planning  space  are  supported.  As  will  be  shown,  this  integrated 
incremental  graph  construction  leads  to  a potentially  significant  reduction  in  the  size  of 
the  graph  by  constraining  the  branching  factor  and  the  total  number  of  nodes  that  must  be 
instantiated  while  the  arc  cost  function  allows  the  soft  constraints  to  elicit  a variety  of 
different  planning  behaviors  from  the  system. 

The  algorithm  consists  of  the  steps  illustrated  in  Figure  4. 

1 ) Insert  rand — - into  open  list  O that  contains  all  visited  nodes. 

7 start  ,t0 

2)  Insert  one  or  more  goal  nodes  rand — into  the  goal  list  G. 

goal,t0  ° 

3)  Using  the  graph  search  technique  of  your  choice,  select  and  remove  a node 
rand  - from  O. 

k.t, 

4)  If  rand^  t e G,  and  re-planning  is  desired  go  to  ( 1 ) else  if  rand^  t e G finished, 
else  go  to  (5). 

5)  Define  the  set  to  be  visited  TBV  = SPrand£  t . 

6)  While  TBV  * 0,  create  arc  ( rand,  ,randT  ) and  evaluate  temporary  variable 

k,t,  Mi+i 

P = pc(prand(  rand — ,rand-  ))  where  rand-  e TBV. 

start. to  *Ai+i  *’4+i 

a.  Remove  rand7  from  the  set  TBV. 

Mi+i 

b.  If  rand7  marked  as  VISITED,  and  R < aT  , set  a7  = fi , set 

Mi+l  *,4+1  Mi+l 

randy  t backpointer  to  rand  - 1 , and  add  randy  t to  O. 

c.  Else  if  randT  not  marked  as  VISITED,  mark  as  VISITED,  set  a-  = B, 

Mi+l  Mi+l 

set  randy  t backpointer  to  rand£  t , and  add  randy  t to  O. 

d.  Remove  arc  (randc,  , rand r ). 

*Mi  Mi+l  7 

e.  End  while. 

7)  Go  to  3. 

Figure  4:  Integrated  incremental  graph  generation  and  planning  algorithm. 

Step  1 and  2 of  the  algorithm  depicted  in  Figure  4 begin  the  search  process  by  placing  the 
node  that  contains  the  current  system  state  into  the  list  O of  open  nodes  and  one  or  more 
goal  states  into  the  goal  set  G.  The  algorithm  will  terminate  when  a minimum  cost  path 
has  been  found  to  any  of  the  nodes  in  the  set  G.  It  should  be  noted  that  the  feature  of 
being  a start  or  goal  node  is  a relevant  attribute  and  therefore  the  start  and  goal  nodes  are 
always  relevant  and  elements  of  RAND.  This  assures  that  the  graph  will  contain  the  start 
and  goal  nodes  even  if  these  nodes  have  no  other  relevant  features  associated  with  them. 
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In  step  3,  it  is  assumed  that  a search  technique  that  utilizes  an  open  list  is  used.  Many 
such  techniques  exist  in  the  current  literature  such  as  breadth  first  [14],  depth  first  [14], 
and  A*  [8].  A search  methodology  dependent  criterion  is  used  to  select  the  node  from  the 
list  O that  should  be  expanded.  If  the  list  O is  empty,  then  a plan  failure  has  occurred.  At 
this  point,  the  system  may  either  return  a failed  status  indicating  that  no  plan  was  found, 
or  may  relax  the  selection  criteria  for  the  NRATT  (i.e.  increase  the  size  of  NRAND)  and 
attempt  to  find  a new  plan.  Through  this  technique,  a hierarchy  of  hard  constraints  may 
be  established  that  are  gradually  relaxed  as  a result  of  plan  failures. 

Step  4 performs  a test  to  determine  if  the  current  node  is  a member  of  the  goal  set  G.  The 
goal  set  is  defined  as  the  set  of  nodes  in  RAND  that  will  satisfy  the  objectives  of  the 
system.  This  set  may  contain  more  then  one  node  (e.g.  a mobile  robot  should  arrive  at  a 
certain  location  as  soon  as  possible,  with  no  specific  time  information  given),  but  should 
never  be  empty. 

Definition  14:  Define  the  goal  set  G as  the  set  of  relevant  annotated  nodes  (RAND)  such 
that  G = { gr  e RAND  | ke  KcS  Ati  e ficT}. 

k,t, 

One  integral  feature  of  this  algorithm  is  dynamic  re-planning.  By  returning  to  step  1 after 
a successful  plan  has  been  generated,  dynamics  in  the  hard  constraints,  soft  constraints, 
and  environment  are  easily  handled.  Additionally,  sections  of  previously  constructed 
plans  are  reused  when  possible  in  order  to  speed  the  re-planning  process. 

Steps  5 and  6 incrementally  expand  and  evaluate  the  graph  structure.  Arcs  are 
temporarily  created  to  link  the  current  node  to  all  of  its  successors,  and  the  path  cost 
function  pc  evaluates  the  new  potential  path  to  the  successor  node.  The  least  expensive 
path  from  a given  node  to  the  start  of  the  search  may  be  found  by  following  a series  of 
backpointers,  and  the  least  expensive  cost  from  a node  to  the  start  of  the  search  is 
maintained  in  the  variable  t . Since  only  a single  path  from  each  node  back  to  the  start 

is  maintained,  the  acyclic  property  of  the  graph  is  guaranteed. 

Graph  Generation  for  various  search  strategies 

Many  different  search  strategies  exist  in  the  literature  that  could  be  used  in  step  (3)  of 
Figure  4 to  determine  the  next  node  to  expand.  These  searches  may  be  decomposed  into 
the  two  broad  categories  of  uninformed  and  informed  search.  The  main  difference 
between  the  two  strategies  is  that  uninformed  search  makes  its  node  expansion  decision 
based  solely  on  the  current  node  and  nodes  on  the  open  list,  while  informed  search  takes 
advantage  of  domain-specific  knowledge  such  as  the  properties  of  the  goal  node. 


Criterion 

Breadth- 

First 

Uniform- 

Cost 

Depth- 

First 

Depth- 

Limited 

Iterative 

Deepening 

Bi-directional 
(if  applicable) 

Time 

bd 

bd 

bm 

b1 

bd 

b*2 

Space 

bd 

bd 

bm 

bl 

bd 

b^2 

Optimal? 

Yes 

Yes 

No 

No 

Yes 

Yes 

Complete? 

Yes 

Yes 

No 

Yes,  if  l>d 

Yes 

Yes 
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Table  1:  Comparison  of  different  search  strategies  (from  Russell  and  Norvig  [14]  p.  81). 
In  this  table,  b is  the  number  of  states  yielded  from  a node  expansion  or  the  branching 
factor , d is  the  depth  of  the  tree,  1 is  the  depth  limit  for  a depth-limited  search,  and  m is 
the  maximum  depth  of  a graph. 

Uninformed  Search 

Table  1 ([14],  p.  81)  provides  information  on  various  properties  about  uninformed  search 
strategies  that  could  be  used  in  step  (3)  to  determine  the  next  node  to  expand.  Information 
is  provided  on  completeness  (the  strategy  is  guaranteed  to  find  a solution  if  one  exists), 
time  complexity  (how  long  the  search  takes  to  find  a solution),  space  complexity  (how 
much  memory  is  required  to  perform  the  search),  and  optimality  (the  strategy  is 
guaranteed  to  find  the  lowest  cost  path).  It  should  be  noted  that  the  memory  requirement 
shown  is  the  amount  of  memory  required  to  maintain  the  list  of  nodes  awaiting  node 
expansion,  and  does  not  include  the  memory  required  to  store  the  graph  structure  itself. 
The  size  of  the  graph  structure  is  the  same  for  all  of  the  search  strategies. 

The  only  precondition  that  exists  for  the  values  given  in  Table  1 is  on  the  cost  function 
used  for  arc  evaluation.  For  the  breadth- first,  iterative  deepening,  and  bi-directional 
(assuming  two  breadth-first  searches)  searches,  optimality  is  only  guaranteed  if  this  cost 
function  is  a non-decreasing  function  of  the  depth  of  a node  in  the  graph.  This  condition 
is  usually  satisfied  only  when  all  arcs  at  a given  depth  have  the  same  cost,  and  is  not 
necessarily  a feature  of  the  cost  of  an  arc  c.  However,  the  breadth-first  search  can  be 
modified  to  become  a uniform-cost  search  that  is  optimal  over  the  cost  of  a path  pc.  For 
the  uniform-cost  search,  optimality  is  guaranteed  if  the  cost  function  is  defined  as 
c:ARC— Since  9?+  is  a totally  ordered  set,  Definition  1 1 allows  this  mapping  as  a 
valid  cost  function. 

Uniform  Cost  Version  of  Algorithm 

Uniform  cost  search  is  a modification  of  the  standard  breadth-first  search  that  finds  the 
cheapest  goal  state  independent  of  its  depth  in  the  graph  (as  computed  by  the  cost 
function  c).  In  order  to  meet  the  criteria  in  Table  1,  the  only  constraint  on  this  form  of 

search  is  that  the  cost  along  a path  must  never  decrease.  In  other  words,  Vi  g N: 
c(r(v,,  vi+1))  > 0 . Once  again,  the  selection  of  c:ARC— >SJT  guarantees  this  condition,  and 
Definition  11  allows  this  mapping.  Therefore,  the  only  change  to  the  generic  graph 
generation  algorithm  presented  above  for  the  use  of  a uniform  cost  search  algorithm  is  in 
step  (3)  of  the  algorithm.  This  step  now  becomes: 

3)  Select  and  remove  a node  cn  from  the  set  O of  open  nodes  such  that: 
cn  e { randc  | V rand  g O:  pc(prand(  rand—  , randc  ))  = 

start, t0  k,tj 

min(pc(prand(  rand— t ,rand))}. 
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Informed  Search 

In  addition  to  the  search  strategies  mentioned  in  Table  1,  many  search  strategies  that  take 
advantage  of  problem  specific  knowledge  exist  in  the  literature  (e.g.  Dijkstra  search  [4], 
A*  [14,17],  bi-directional  A*  [10],  and  D*  [16]).  This  class  of  search  strategies  is  known 
as  informed  search.  The  class  of  informed  strategies  that  utilize  parameters  of  the  goal 
node  is  known  as  best-first  search.  This  name  originates  from  the  fact  that  these 
algorithms  utilize  an  evaluation  function  that  returns  the  estimated  desirability  of 
expanding  a node,  and  hence  the  “best”  node  is  expanded  first.  As  in  the  uninformed 
search,  there  are  informed  search  strategies  that  are  optimal  and  guaranteed  to  find  a 
solution  if  one  exists  and  those  that  are  not.  One  basic  best-first  search  technique  that  is 
guaranteed  optimal  and  complete  [14]  is  the  A*  search.  A*  search  may  be  written  as  a 
function  f(n)  = g(n)  + h(n)  where  g(n)  is  a function  that  computes  the  cost  of  reaching 
node  n from  the  start  node  and  h(n)  is  a heuristic  function  that  estimates  the  cost  of 
achieving  the  goal  node  from  n.  To  be  optimal  and  complete,  this  search  requires  that  the 
function  h(n)  that  estimates  the  desirability  of  expanding  a node  n never  underestimates 
the  desirability  of  the  goal  (or  never  overestimates  the  remaining  cost  to  the  goal)  and  a 
search  using  only  g(n)  is  optimal  and  complete.  The  uniform  cost  search  strategy  was 
shown  to  be  optimal  and  complete  and  may  be  used  for  g(n).  Therefore,  if  h(n)  is  an 
admissible  heuristic  (never  overestimates  the  cost  to  reach  the  goal)  A*  will  be  optimal 
and  complete  for  the  graph  structure  defined  above. 

A*  Version  of  Algorithm 

A*  search  is  a modification  of  the  uniform  cost  search  presented  in  Figure  4.  Rather  then 
expanding  the  cheapest  node  from  the  open  list,  it  expands  the  node  whose  cost  plus  the 
estimated  remaining  cost  to  the  goal  is  cheapest.  Therefore,  the  only  change  to  the 
generic  graph  generation  algorithm  presented  above  for  the  use  of  an  A*  search  algorithm 
is  in  step  (3).  This  step  now  becomes: 


3)  Select  a node  cn  to  open  from  the  set  O of  open  nodes  such  that: 
cn  = rand-  : V rand:  e O,  pc(prand( rand — , rand-  ) + h( randc  ) < 

k,t,  j ,tj  start, t0  k,t,  k,tj 

pc(prand(  rand — - , rand^  ) + h(  rand  )),  where  h()  is  an  admissible  heuristic  function. 

start, tg  J»H  J?tj 


Example  of  framework  applied  to  Towers  of  Hanoi 

In  order  to  illustrate  the  integrated  incremental  graph  generation  and  evaluation 
algorithm,  let  us  examine  the  basic  game  of  “Towers  of  Hanoi”  as  a simple  component 
assembly  problem.  In  this  game,  the  player  is  given  three  towers,  one  of  which  contains  n 
different  sized  stacked  rings  (the  sub-assembly  inventory).  The  objective  of  the  game  is 
to  move  all  of  the  rings  to  the  rightmost  tower  (the  final  product).  The  only  rules  are  that 
you  may  only  move  one  ring  at  a time,  and  must  never  allow  a ring  to  rest  upon  a smaller 
ring.  Typically,  the  starting  configuration  is  to  have  all  of  the  rings  on  the  leftmost  tower. 
However,  this  starting  configuration  may  be  varied. 
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Figure  5:  Illustration  of  Towers  of  Hanoi  with  four  rings. 


As  shown  in  Figure  5,  a system  that  contains  at  most  n rings  will  have  three  towers  each 
containing  n ring  positions.  In  the  naive  system,  where  the  exact  number  of  rings  (in  both 
total  quantity  and  number  of  each  type)  is  not  known,  all  possible  combinations  of  rings 
must  be  mapped  into  nodes.  The  reason  for  this  is  that  a minimum  set  of  basic  rules  is 
assumed  for  the  creation  of  the  planning  graph,  and  all  remaining  game  knowledge  is 
contained  in  the  arc  evaluation.  This  produces  (»  + Impossible  different  individual  tower 
configurations  or  ( n + if"  nodes  for  the  entire  system.  Using  traditional  graph  approaches, 

o 

this  will  produce  a graph  that  contains  *2.4x10  nodes  for  a system  of  up  to  four  rings. 
The  branching  factor  for  such  a naive  system  will  be  (3n-l)(2n)+n  or  92.  This  number 
comes  from  the  fact  that  each  ring  can  move  to  1 of  1 1 different  positions  (3n- 1 ),  each 
ring  may  be  added  in  each  of  1 1 positions  (3n-l),  and  each  ring  may  be  deleted  (n).  As 
shown  in  Table  1,  this  branching  factor  directly  impacts  both  the  time  and  memory 
complexity  of  the  search  algorithms.  Even  in  this  simple  example  the  problem  of  an 
unreasonable  number  of  possible  graph  nodes  and  connections  is  quickly  reached.  Yet 
given  this  unreasonable  number  of  nodes  and  connections,  the  system  is  still  unable  to 
cope  with  unexpected  environmental  changes.  The  reason  for  this  is  that  the  graph  is  built 
before  the  planning  process  begins  and  only  anticipated  actions  are  built  into  the  graph. 


Figure  6:  Description  of  various  attribute  sets. 
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In  order  to  reduce  the  number  of  nodes  and  edges  to  a reasonable  level,  the  integrated 
incremental  graph  generation  and  evaluation  algorithm  will  be  applied.  It  will  be  shown 
that  this  algorithm  may  significantly  reduce  the  number  of  instantiated  nodes  and  will 
reduce  the  branching  factor  to  3,  regardless  of  the  number  of  rings.  This  algorithm  will 
also  allow  environmental  dynamics  to  be  represented  and  evaluated  in  the  planning 
graph. 

Before  the  graph  generation  and  evaluation  algorithm  can  be  applied,  the  members  of  the 
attribute  sets  must  be  defined.  For  this  particular  problem,  NGATT  contains  the  four 
attributes  of  obeying  gravity  (spaces  are  not  allowed  between  the  rings  on  a tower),  total 
number  of  rings,  number  of  size  i rings,  and  obeying  the  rules  of  the  game  (the  plan  is  not 
allowed  to  cheat).  As  shown  in  Figure  6,  NGATT  contains  all  possible  subsets  of  the 
attributes.  This  set  corresponds  to  the  set  of  nodes  that  would  be  found  in  a traditional 
planning  graph.  The  set  NATT  c=  NGATT  restricts  the  node  space  to  obey  gravity  and  to 
contain  one  each  of  three  different  sized  rings.  The  set  NRATT  cz  NATT  further  restricts 
the  node  space  to  force  compliance  with  the  rules  of  the  game. 


Optimal  Path 


AjIA'I  iIS'1  A 


Figure  7:  Optimal  path  found  using  uniform  cost  function  and  nodes  RAND.  In  the 
diagram,  a single  line  (|)  separates  successors  of  a given  node  and  a double  line  (||) 
separates  one  node’s  successors  from  another’s.  Arcs  are  shown  as  a dotted  line. 


Figure  7 illustrates  the  optimal  path  found  by  the  graph  generation  and  evaluation 
algorithm.  It  should  be  noted  that  a traditional  graph  would  contain  262,144  nodes.  These 
nodes  would  also  have  to  be  connected  by  arcs.  Even  if  there  was  only  one  arc  per  node 
(making  graph  search  unnecessary)  there  would  be  262,144  arcs.  By  using  relevant 
attributes,  the  number  of  nodes  can  be  reduced  to  only  27  nodes  and  64  arcs.  This  low 
number  is  due  to  the  branching  factor  being  reduced  to  3 by  the  constraints  imposed  by 
NRATT.  In  addition  to  this  low  branching  factor,  many  of  the  arcs  are  never  created  due 
to  their  evaluated  cost  being  higher  or  equal  to  an  existing  arc.  This  provides  a savings  of 
at  least  4 orders  of  magnitude  of  the  memory  required  to  store  the  graph.  In  addition  to 
the  memory  savings,  the  only  nodes  that  need  to  be  evaluated  for  arc  generation  are  the 
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27  nodes  that  were  touched  by  the  search.  In  the  traditional  search  approach,  all  262,144 
of  the  nodes  would  have  been  evaluated  for  arc  connections  before  the  search  begins. 


£ I l-AA  l-l  £4-1  1^-1  I & 

Figure  8:  Towers  of  Hanoi  solution  where  the  rules  of  the  game  were  not  considered 
relevant. 


Figure  9:  Towers  of  Hanoi  solution  where  the  rules  of  the  game  concerning  which  rings 
may  be  moved  became  relevant,  but  the  legality  of  the  actual  tower  configuration  was 
ignored. 


Rule 

4 I 1-4  I 1-4  A I-'" 4 I 1-4  I 1-4  A 1-44  I 

(-1  1,4-1  A4-I  1 4— | 14 

Figure  10:  Towers  of  Hanoi  solution  where  the  rules  of  the  game  became  relevant  after 
the  start  of  plan  execution,  and  existing  illegal  tower  configurations  were  discouraged. 


As  shown  in  Figure  8 through  Figure  10,  the  behavior  of  the  system  may  be  altered 
through  the  selection  of  the  relevant  attributes.  In  Figure  8,  NRATT  = NATT  and  arcs 
representing  all  possible  moves  (both  legal  and  illegal)  are  created  to  connect  the 
members  of  NRAND  as  the  graph  is  expanded.  Figure  9 displays  the  case  where  the 
relevant  rules  were  changed  in  the  middle  of  plan  execution,  and  a replanning  cycle 
occured.  NRATT  started  as  NRATT  = NATT  and  was  changed  after  two  moves  were 
executed  to  the  reduced  set  which  requires  ATT4  to  be  kktrue”.  In  this  case,  the  only  arcs 
that  are  created  represent  legal  future  moves.  Since  the  current  state  of  the  system  is  an 
illegal  state  (by  the  new  NRATT  definition)  NRAND  will  contain  illegal  states. 

However,  all  planned  ring  moves  will  be  legal.  In  contrast,  NRATT  in  Figure  10  provides 
for  only  legal  arcs  to  be  generated  and  has  the  additional  constraint  implemented  in  the 
cost  function  that  illegal  states  are  more  expensive  then  legal  states.  Upon  replanning 
with  the  new  rules  and  cost  function,  this  causes  the  system  to  quickly  undo  the  illegal 
move  and  put  the  system  in  a legal  state  before  proceeding  with  the  problem  solution.  It 
should  be  noted  that  while  the  system  plans  the  optimal  set  of  node  transitions  from  the 
current  time  forward,  dynamics  might  cause  the  global  path  from  the  initial  state  to  be 
sub-optimal.  As  shown  in  Figure  10,  previously  performed  moves  must  be  undone  thus 
creating  a longer,  sub-optimal  final  path. 

The  rule  changes  depicted  in  Figure  9 and  Figure  10  occurred  after  the  system  had 
executed  two  moves.  If  these  rule  changes  could  have  been  anticipated  through  the  use  of 
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the  time  dimension  of  NRATT,  then  the  plan  for  Figure  9 would  remain  unchanged,  and 
the  plan  for  Figure  10  would  be  the  same  as  the  plan  from  Figure  7.  The  reason  for  this  is 
that  the  system  would  be  able  to  anticipate  the  need  to  undo  moves  and  therefore  would 
not  make  them  in  the  first  place. 

Parallel  planning  in  partitioned  problem  spaces 

While  the  set  of  possible  general  node  attributes  (NGATT)  is  the  same  over  the  entire 
node  space  ND,  in  many  practical  applications  the  possibility  of  certain  node  attributes 
occurring  may  be  tied  to  local  phenomena.  It  is  also  possible  that  the  location  of  these 
local  phenomena  is  tied  to  a subset  of  the  possible  general  node  attributes 
NATTr  cz  NATT  c NGATT  . The  set  of  relevant  attributes  that  are  selected  for  use  in 

graph  construction  will  still  be  of  the  form  NRATTr  c=  NATTr  . A region  may  then  be 
defined  as  a subset  of  the  nodes  in  RAND  that  share  certain  relevant  attributes. 

Definition  15:  A region  R!  is  defined  as  a subset  Rn  of  the  relevant  annotated  nodes 
RAND  where  R,  = { randt  ( | ||nratt£‘t  ||  > 0,  k = (k15...,kn),  ki,...,kn  e N,  1 = (l1;...,ln) , 

li In  e N,  t,  e T }. 

To  complete  the  definition  of  a region,  a set  of  regions  that  contains  all  of  the  nodes  form 
RAND  will  be  defined  as  a complete  set  of  regions. 

n 

Definition  16:  The  set  of  n regions  R = {Ri,...,Rn  } is  complete  iff  [JR;  = RAND . 

i=l 

It  can  be  shown  that  any  node,  nd  e RAND  is  also  a member  of  the  complete  set  of 
regions  R. 


Lemma  2:  If  nd  e RAND  and  R = {Ri,  ...,  Rn}  is  a complete  set  of  regions,  then  nd  e R. 

n 

Proof:  Assume  nd  g R,  then  by  Definition  16,  R = (JR,  = RAND  and  nd  € RAND 

i=l 

which  contradicts  the  original  assumption. 

One  common  occurrence  of  regions  is  in  the  decomposition  of  a planning  space  into  a 
hierarchy.  Figure  1 1 shows  one  possible  technique  for  forming  a hierarchy  that  was 
presented  by  Fernandez  and  Gonzalez  [5,7]  where  high-resolution  nodes  are  abstracted  to 
form  a smaller  number  of  lower  resolution  super  nodes.  Each  super  node  is  then 
connected  to  form  a planning  graph  and  planning  is  performed  on  this  low-resolution 
representation  of  the  space.  Fernandez  and  Gonzalez  present  a technique  for  connecting 
and  searching  these  graph  structures  that  is  based  on  the  classic  refinement  method. 
However,  the  authors  state  that  this  technique  is  not  guaranteed  to  find  the  cost  optimal 
path  ([7],  p.  106).  In  the  refinement  method,  a path  is  first  found  through  the  low- 
resolution  graph.  In  time-constrained  environments,  this  path  may  be  executed  as  a sub- 
optimal  plan.  If  more  time  is  available,  individual  super  nodes  that  are  part  of  this  path 
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are  expanded  and  a new  refined  plan  is  constructed  that  is  a refinement  of  the  previous 
path.  Any  number  of  hierarchical  levels,  referred  to  as  a multi-hierarchy,  may  be 
imbedded  in  such  a representation  and  the  multi-hierarchy  may  act  as  an  anytime 
algorithm  [18]. 


(c) 


Figure  11:  Example  from  Fernandez  and  Gonzalez  ([7],  p.104)  of  a graph-based 
abstraction  hierarchy,  (a)  world  view  of  a room,  (b)  hierarchical  graph  model  with  levels 
represented  by  shaded  regions  as  keyed  by  (c). 


Another  way  to  view  the  above  hierarchy  [5]  and  multi-hierarchy  [7]  is  through  the  use  of 
regions  as  defined  above  and  shown  in  the  left  hand  side  of  Figure  14.  Under  this  view, 
the  set  RAND  would  contain  all  of  the  Level  0 nodes  and  would  be  defined  by  the  set  of 
node  relevant  attributes  NRATT.  A super  node  is  then  defined  as  a region  c=  RAND 
such  that  NRATTr  c=  NRATT . For  planning  purposes,  individual  nodes  may  then 

represent  these  regions,  and  a regional  spanning  distance  may  then  be  defined  according 
to  Definition  8 that  could  be  used  in  the  integrated  incremental  graph  creation  and 
evaluation  algorithm.  Modifications  to  the  integrated  incremental  graph  creation  and 
evaluation  algorithm  that  will  guarantee  that  the  cost  optimal  path  is  found  are  presented 
below. 

Another  view  of  hierarchical  planning  is  presented  by  Lacaze  [9]  for  mobile  robot 
applications.  In  this  paper,  the  planning  space  is  viewed  as  having  a high  node  density 
near  the  robot  with  decreasing  density  as  the  distance  to  the  robot  increases.  Forming 
regions  of  equal  node  count  as  shown  in  Figure  12  creates  the  hierarchy.  A region  of  high 
node  density  exists  around  the  robot’s  current  location  where  high-resolution  sensor 
information  is  available.  Further  from  the  robot,  the  sensors  are  only  able  to  detect  large 
objects  and  the  corresponding  node  density  decreases.  Finally,  once  the  limits  of  the 
sensors  have  been  exceeded,  the  node  density  drops  to  correspond  to  the  feature  density 
represented  in  the  a priori  dataset. 
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Figure  12:  Example  of  node  density  for  2-D  mobile  robot  application  based  on  Lacaze 
[9].  Rings  represent  regions  of  equal  node  count. 


The  general  approach  of  regions  can  once  again  be  specialized  to  account  for  this  form  of 
hierarchy.  As  an  example  of  this,  examine  the  mobile  robot  application  with  NGATT 
composed  of  high-resolution  sensed  feature  information  and  low-resolution  a priori 
feature  information.  The  high-resolution  information's  availability  is  tied  to  the  local 
phenomena  of  distance  from  the  location  of  the  robot’s  sensors,  while  the  low-resolution 
features  are  available  over  the  entire  map  extents.  This  problem  may  be  naturally 
partitioned  into  a region  that  contains  high-resolution  information  (Ri)  and  a region  that 
does  not  (R2)  where  NATTR  2 NATTR  . Since  NATTR  id  NATTr  , it  is  possible  that 

more  attributes  are  available  for  selection  as  members  of  NRATT,  and  that  the 
composition  of  RAND  in  the  region  Ri  defined  by  RATTr  will  be  different  than  in  the 
region  R2  defined  by  RATTR2 . 

Once  a density  pattern  such  as  the  one  in  Figure  12  has  been  detected,  the  node  space 
may  be  naturally  partitioned  into  regions  where  the  predicted  node  count  and  graph 
search  complexity  will  be  approximately  equal.  Responsibility  for  planning  in  each 
region  may  then  be  partitioned  to  separate  processes,  and  these  processes  communicate 
through  the  use  of  a graph  construction  technique  such  as  the  one  presented  by  Lacaze 

[9]- 

Lacaze  [9]  describes  this  technique  for  use  in  hierarchical  planning  where  regions  of 
decreasing  node  density  exist  in  concentric  areas  that  originate  at  the  system's  current 
location  (much  as  the  concentric  circles  depicted  in  Figure  12)  and  the  levels  of  the 
hierarchy  plan  at  different  rates.  The  only  additional  precondition  for  the  use  of  this 
technique  is  that  the  reverse  arc  cost  function  c'  must  exist. 


Definition  17:  Define  the  reverse  arc  cost  function  c'  as  the  mapping  c':  ARC— >tR+ 1 c( 
arc(rand^t,  randr  t+d ) ) = c'(  arc(randT  t+d , rand^)),  k = (k1?...,kn),  ki,...,kn  e N, 
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1 = (lj,...,ln) , li, ,ln  e N,  d e N,  f e T.  Also,  define  the  reverse  cost  associated  to  a 

m 

path  prand  as  pc'  =Zc'wvi’vw»- 

i=l 

This  cost  function  is  necessary  in  the  algorithm  implemented  by  Lacaze  [9]  because  he 
plans  from  the  goal  node  to  the  start  node.  In  the  following  sections,  Lacaze’s  technique 
[9]  will  be  formalized,  and  it  will  be  shown  that  a generalization  of  this  technique  may  be 
viewed  as  a specific  implementation  of  the  planning  framework  described  in  this  paper. 
Lacaze's  technique  will  then  be  extended  to  apply  to  the  more  general  case  of  an  arbitrary 
number  of  regions  with  arbitrary  boundaries. 

The  algorithm  described  below  plans  from  the  start  node  to  the  goal  node,  however  goal 
to  start  planning  may  be  performed  by  exchanging  the  start  and  goal  nodes  and  using  the 
cost  function  pc'.  The  path  obtained  through  this  exchange  will  have  the  same  minimum 
cost,  but  may  contain  different  nodes.  The  reason  for  this  is  that  the  search  algorithm  is 
complete  and  optimal  and  will  give  a path  with  a cost  equivalent  to  the  minimum  cost 
path  from  start  to  goal.  However,  if  more  then  one  equivalent  minimum  cost  path  exists 
from  start  to  goal,  different  paths  may  be  found  by  the  two  approaches. 

During  the  operation  of  the  planning  algorithm  that  is  described  in  Figure  13,  it  is 
important  to  know  if  a node  is  on  the  boundary  or  in  the  interior  of  a given  region.  A 
node  will  be  defined  to  be  on  the  boundary  of  regions  Ra  and  Rb  if  it  is  a member  of  both 
regions  and  its  spanning  set  includes  at  least  one  node  that  is  an  exclusive  member  of 
each  region. 


Definition  18:  Given  the  set  of  n regions  Ri,...,Rn  , the  boundary > set  of  any  two  regions 
i,j  is  defined  as  By  = { randk  t e R„Rj  | 3 node  randy  t e SP(  rand- 1 ) such  that  randy  t £ R, 

and  randy  ( e R,  A 3 node  randy  t e SP(  randk  t ) such  that  randy  t e Rj  and  randy  t g Rj  }. 

Two  regions  Ri,  Rj  will  be  denoted  as  shared  boundary  regions  if  their  intersection  is 
equal  to  the  boundary  region  between  the  two  sets.  Shared  boundary  regions  have 
“shallow”  boundary  regions  that  minimize  the  region  overlap. 


Definition  19:  Two  regions  Ri,  Rj  are  called  shared  boundary’  regions  SBR!  and  SBRj  iff 

R,  n Rj  = By  = Bj  i . 

In  order  to  specialize  the  integrated  incremental  graph  generation  and  planning  algorithm 
from  Figure  4,  a new  notion  of  a spanning  set  that  is  specialized  for  regions  must  be 
defined.  It  will  be  shown  that  the  union  of  these  regional  spanning  sets  is  equal  to  the 
spanning  set  defined  in  Definition  8. 
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Definition  20:  Define  the  regional  spanning  set  SPR,  of  a node  rand- 1 as  the  set  of  all 
successor  nodes  SPRj  (rand- 1 ) = { sp^  t+]  | |j  nratt-  ||  > 0 a sp-  e Rj}. 


Lemma  3:  V rand^  t e RAND,  SP(rand^  t ) = (JSPR^rand^  t)  where  R=  {Ri,  Rn}  is 

a complete  region. 

Proof: 

Definition  20  may  be  rewritten  as  [JSPRy  ( rand- 1 ) = { sp^  t M | ||  nratt- 1 x ||  > 0 a sp-  e 
IjR,  }.  From  Definition  16  and  the  fact  that  R is  complete,  (^JSPRy  ( rand^  ) = { sp£  | 

i i 

||  nratt- 1+1 1|  > 0 a sp^  t+1  e RAND}.  By  Definition  6,  the  condition  sp-  e RAND  is 
redundant,  and  the  definition  reduces  to  (^JSPR1  (rand^  t ) = { sp-  t+l  | ||  nratt^  (+1 1|  > 0}  = 

i 

SP(  rand- 1 ) by  Definition  8. 


Planning  algorithm  for  SBRr  where  R={Ri, ...,  Rn}.  All  nodes  start  as  not  VISITED. 

1)  If  region  contains  rand—  , insert  this  into  region's  open  list  Or  that  contains  all  VISITED  nodes. 

start, t0 

2)  If  region  contains  one  or  more  goals,  rand — - , insert  them  into  Gr. 

goal,t0 

a)  Form  the  set  G*  = Gr  u B(rj),  j g {1,  ...,  n}. 

3)  Using  the  graph  search  technique  of  your  choice,  select  and  remove  a node  rand- 1 from  the  Or. 

4)  If  randy  t g Gr,  go  to  (2). 

a)  Else,  if  randy  t g G*,  remove  randy  t from  G*.  If  G*  = 0,  go  to  (2). 

5)  Define  the  set  to  be  visited,  TBVr  = SPrandy  t . 

6)  While  TBVr  ^ 0.  create  arc  ( rand-  ,rand-  ) and  evaluate  temporary  variable 

Mi  ,»ti+2 

P = pc(prand(rand— ( ,randft  ))  where  randy  t g TBVr. 

a.  Remove  rand  T from  the  set  TBVr. 

* »li+l 

b.  If  rand T marked  as  VISITED,  and  p < aT  , set  aT  = p,  set  rand T backpointer  to 

1 1 ,tj+i  I ,tj+i  1 ,tj  +1 

randy  t , and  add  randy  t to  Or.  If  randy  t g B(r,j),  r * j,  j e { 1,  ...,  n},  add  randy  ( | to  Oj. 

c.  Else  if  rand,  not  marked  as  VISITED,  mark  as  VISITED,  set  aT  = p,  set 

randr  backpointer  to  rand.-  , and  add  rand  T to  Or. 

d.  Remove  arc  ( rand,-  ,randT  ) 

Mi  Mi+i 

e.  End  while. 

7)  Go  to  3. 


Figure  13:  Integrated  incremental  graph  generation  and  planning  algorithm  specialized 
for  planning  in  partitioned  spaces.  Each  SBR  executes  its  own  version  of  the  algorithm. 
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Definition  21:  Define  a complete  cycle  as  the  execution  of  step  6 of  the  algorithm  of 
Figure  13  without  changing  any  a values. 

Figure  13  describes  a specialization  of  the  integrated  incremental  graph  generation  and 
planning  algorithm  from  Figure  4 for  operation  over  a partitioned  planning  space.  A 
separate  version  of  this  algorithm  will  be  executed  for  each  of  the  partitions  R,  e R,  i = 

{1,  ...,  n}.  These  algorithms  may  be  run  serially,  or  may  be  executed  in  parallel  with  no 
effect  on  the  final  complete  solution  as  will  be  shown  in  Theorem  1 . The  only  change  to 
the  basic  framework  that  is  necessary  for  operation  over  partitions  concerns  the  inclusion 
of  the  boundary  nodes  into  the  algorithm  framework.  These  boundary  nodes  are  made 
part  of  the  goal  set  for  the  partition,  and  must  be  reached  in  regions  that  do  not  include 
the  goal  before  the  algorithm  can  terminate. 

In  order  to  include  the  boundary  as  part  of  the  goal  set,  step  (2)  forms  the  new  goal  set  G* 
that  includes  the  systems  goals  as  well  as  the  boundary  regions.  One  of  the  main  features 
of  the  algorithm  is  that  the  path  cost  from  a boundary  node  nd  e B(R1?  Rj)  to  the  start 
node  may  be  changed  by  the  search  algorithm  running  in  either  region  Rj  or  Rj.  In  order 
to  properly  propagate  this  value  to  the  region  not  responsible  for  the  value  change,  the 
stopping  criterion  in  step  (4)  must  be  changed  to  cause  repeated  planning  cycles  to  occur. 
The  repeated  cycle  will  quickly  terminate  (no  nodes  other  then  the  boundary  nodes  will 
be  placed  on  the  open  list)  if  no  changes  have  occurred.  However,  if  boundary  changes 
have  occurred,  they  will  propagate  through  the  graph. 


Figure  14:  Set  relationships  for  partitioned  planning  space. 

The  algorithm  from  Figure  13  may  be  applied  to  any  configuration  of  shared  boundary 
regions.  Two  possible  configurations  are  shown  in  Figure  14,  where  the  left  hand 
drawing  represents  a hierarchy  of  the  form  of  Fernandez  and  Gonzalez  [7],  and  the  right 
hand  drawing  represents  a hierarchy  of  the  form  of  Lacaze  [9].  In  general,  these  partitions 
may  exist  anywhere  in  the  planning  space  with  a separate  graph  search  system  operating 
on  each  partition.  If  the  individual  plans  constructed  in  each  partition  are  optimal,  then 
the  final  plan  constructed  by  the  algorithm  of  Figure  13  will  also  be  optimal. 

Theorem  1:  Given  the  boundary  regions  R = {Ri,  ...,  Rn}  with  a independent  version  of 
the  planning  algorithm  from  Figure  13  executing  in  each  region  using  an  algorithm  that  is 
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complete  on  a standard  planning  space  for  node  selection.  Further,  given  the  start 
node  rand- 1 , and  goal  node  rand-  t the  path  prand(  rand^  t , rand-  t +m ) is  produced 

once  all  algorithms  have  finished  a complete  cycle.  Then  prand(  rand-  t ,randg  t +m ) = 
prandmm(  rand- 1 , rand- 1 +m ) is  the  cost  optimal  path  through  the  graph  in  the  regions  R, 
and  it  is  the  same  path  as  the  cost  optimal  path  through  the  graph  in  RAND. 

Proof  (a):  Assume  prand(rand- 1 ,rand-  t +m)  * prandmm(  rand-  t ,rand- 1 +m). 

Then  3 prandmm*(  rand- 1 ,randiti+m) : pc(prandmm*(  rands  t ,rand-  t +m))  < 
pc(prandmm(  rand- 1 , rand- t.  +m )). 

By  Definition  9 3 node  nd  : nd  g R,  nd  G prandmm(  rand- 1 ,rand- 1 +m)  a 
pc(prandmin*(  rand§  t , nd))  + pc(prandmm*(nd,  randg  t +m ))  < 
pc(prandmm(  rand- 1 ,randg  t ))  where  nd  is  the  first  node  from  the  list  V:  Vj  g 
V(prandmm*)  a vj  G prandmm(  rand- 1 ,randi  t +m). 

Without  loss  of  generality,  assume  that  vj.)  g R,. 

Four  different  situations  exist  with  respect  to  boundary  set  membership  for  the  nodes  nd 
and  Vi-i. 

Case  1 : vm  ^ B(R„  Rj)  a nd  £ B(R„  Rj). 

Case  2:  v^i  ^ B(R„  Rj)  a nd  g B(R„  Rj). 

Case  3:  vi_i  g B(R„  Rj)  a nd  ^ B(R,,  Rj). 

Case  4:  Vj.i  g B(R„  Rj)  a nd  g B(R,,  Rj). 

For  case  1 and  case  4,  both  vm,  nd  g R„  i g {1,  ...,  n}.  In  these  cases,  both  nodes  are  in 
the  same  region  and  a cost  optimal  algorithm  is  running  in  that  region.  The  fact  that  nd  is 
not  included  in  the  final  path  contradicts  the  assumption  that  the  individual  region’s 
algorithm  is  cost  optimal. 

For  case  2,  nd  g Rj,  nd  G SPR  (v^) , i g j g {1,  ...,  n}.  By  Definition  17,  3 arc(  vm,  nd), 
nd  g SPRj(Vl). 

Therefore  either  nd  is  on  open  list  Oj,  or  nd  has  already  been  expanded.  Since  given  that 
algorithm  cycle  complete,  all  nd  | pc(prand(  rand- 1 , nd))  < 

pc(prand(  rand- 1 ,rand- 1 +m  ))GO,.  This  forces  the  conclusion  that  nd  has  been  expanded. 

However,  if  nd  has  been  expanded,  then  by  step  6 of  algorithm,  prandmm  would  include 
this  node.  This  contradicts  the  initial  assumption. 


For  case  3,  vm  g R„  nd  g Rj,  Rj,  nd  g SPr  (v^)  , i ^ j e {1,  ...,  n}.  By  Definition  17,  3 
arc(  vm,  nd),  nd  g SPr  (vm)  . 

Therefore  nd  on  open  list  or  expanded  for  region  R.  If  nd  is  expanded  in  region  R„  then 
by  step  6 of  algorithm  nd  will  either  be  on  Oj,  or  prandmm  would  include  this  node. 

prandmm  including  this  node  contradicts  the  initial  assumption.  Therefore  nd  g O,  V 0,. 
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However,  since  given  that  algorithm  cycle  complete,  all  nd  | pc(prand(  rand- 1 , nd))  < 
pc(prand(  rand§  t ,rand- 1 +m  ))£0„  Oj.  This  forces  the  conclusion  that  nd  has  been 

expanded  in  both  regions  which  contradicts  nd  e O,  V Oj. 

Proof  (b):  From  Lemma  2 and 

Lemma  3,  every  node  and  arc  that  exist  in  Gr  also  exists  in  Grand- 

Therefore  all  paths  that  exist  in  R exist  in  RAND,  and  all  paths  that  exist  in  RAND  exist 

inR. 

As  previously  detailed,  one  common  form  of  shared  boundary  region  is  a hierarchy.  The 
hierarchy  from  the  mobile  robot  example  is  covered  by  the  general  algorithm  presented 
above  by  constructing  the  complete  set  of  regions  R such  that  Ri  and  Rn  have  only  one 
boundary  region  (with  R2  and  Rn-i  respectively)  and  all  intermediate  regions  i have  two 
regions  (one  with  R,_i  and  one  with  R1+1).  A three  level  hierarchy  of  this  form  is  shown  on 
the  right  hand  side  of  Figure  14. 

Example  of  framework  applied  to  mobile  robot 

Differences  in  the  availability  of  environmental  data  may  not  be  the  only  difference 
between  the  RAffL  subsets  for  the  different  levels  of  a planning  hierarchy.  Differences  in 
planning  objectives  for  the  levels  may  dictate  that  other  constraints  are  to  be  placed  on 
the  selection  of  RATT.  For  example,  the  highest  resolution  level  of  planning  may  be 
concerned  with  creating  the  optimal  dynamically  correct  path  while  a lower  resolution 
level  is  concerned  with  computing  the  optimal  set  of  roads  to  traverse  to  reach  the 
objective.  The  fact  that  a path  must  be  dynamically  feasible  to  be  considered  is  defined  as 
a hard  constraint  that  is  formally  defined  by  Definition  22.  These  hard  constraints  may  be 
seen  as  not  only  controlling  which  attributes  are  possible  for  nodes  to  contain,  but  also  as 
shaping  the  topology  of  the  graph  by  controlling  which  nodes,  and  therefore  which  node 
transitions  are  possible. 

Definition  22:  A hard  constraint  is  defined  as  an  element  of  NRATT  where  each 
attribute  is  defined  by  a predicate. 


In  addition  to  the  constraints  that  are  placed  on  the  selection  criteria  for  the  inclusion  of 
attributes  into  NRATT,  the  inclusion  of  nodes  into  a path  that  contain  certain  attributes 
may  be  more  desirable  than  the  inclusion  of  nodes  that  do  not  contain  these  attributes. 
This  notion  of  desirability  may  be  formalized  by  the  notion  of  soft  constraints.  Soft 
constraints  are  implemented  through  the  arc  cost  function  and  control  a cost  ranking  that 
signifies  which  arcs  are  more  desirable. 

Definition  23:  A soft  constraint  is  defined  as  an  ordering  placed  on  the  arcs  by  the  cost 
function  c. 
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Relevant  Attribute  Number  and 

Description 

NRATT1:  Steering  curvature 
NRATT2:  High-resolution  features 
NRATT3:  Location  L 
NRATT4:  A priori  features 
NRATT5:  Expected  congestion 


NRATT 


NRATT1;  e 9? 

NRATT2:  Obstacles,  road  edges, 
NRATT3:  Any 

NRATT4:  Roads,  buildings, ... 
NRATT5:  7-9  am  high,  else  low 


NRATT 


^ NRATT-.  x 
NRATT3:  L2 
NRATT4:  All 
NRATT5:  Low 


NRATT1:  < a 
NRATT2:  All 
NRATT3:  Li 


Figure  15:  Example  of  attribute  subsets  used  to  generate  regions  for  a hierarchical  mobile 
robot  application. 


Constraint  Set 

Constraint  Set  2 

Constraint 

Cost/Meter 

Constraint 

Cost/Meter 

NRATT4  = Highway  a 
NRATT5  * Low 

50.0 

NRATT4  = (Highway  v Minor 
Road)  a NRATT5  * Low 

100.0 

NRATT4  = Minor  Road 

1.0 

NRATT4  = Offroad 

2.0 

NRATT4  - Highway  a 
NRATT5  = Low 

1.5 

NRATT4  = (Highway  v Minor 
Road)  a NRATT5  = Low 

1.0 

NRATT4  * Highway  v 
Minor  Road 

200.0 

Figure  16:  Possible  soft  constraint  sets  based  on  NRATT  from  Figure  15  for  region  R2. 


Examples  of  hard  constraints  may  be  seen  in  Figure  15  while  two  different  sets  of  soft 
constraints  for  the  cost  function  are  shown  in  Figure  16.  The  hard  constraints  for  region 
R2  will  assure  that  no  path  is  ever  considered  that  contains  areas  of  high  traffic 
congestion.  The  soft  constraints  for  constraint  set  1 favor  travel  on  minor  roads  (even  if 
the  path  is  longer)  and  severely  penalize  path  segments  that  lie  off-road.  For  constraint 
set  2,  off-road  is  twice  as  expensive  as  on-road  travel,  while  travel  on  congested 
roadways  is  penalized.  This  combination  of  hard  constraints  through  planning  regions 
coupled  with  soft  constraints  through  the  arc  cost  function  provides  the  integrated 
incremental  graph  creation  and  evaluation  algorithm  a large  degree  of  flexibility  in  how 
plans  may  be  constructed  to  satisfy  the  system’s  objectives. 
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